Skip to content

[CI] (4ac81e3) fastapi/fastapi3-ai-saas - #2453

Closed
wizard-ci-bot[bot] wants to merge 1 commit into
mainfrom
wizard-ci-4ac81e3-fastapi-fastapi3-ai-saas
Closed

wizard-ci-bot[bot] wants to merge 1 commit into
mainfrom
wizard-ci-4ac81e3-fastapi-fastapi3-ai-saas

Conversation

@wizard-ci-bot

@wizard-ci-bot wizard-ci-bot Bot commented Jul 7, 2026

Copy link
Copy Markdown

Automated wizard CI run

Source: wizard-pr
Trigger ID: 4ac81e3
App: fastapi/fastapi3-ai-saas
App directory: apps/fastapi/fastapi3-ai-saas
Workbench branch: wizard-ci-4ac81e3-fastapi-fastapi3-ai-saas
Wizard branch: fix/pi-harness-use-warlock
Context Mill branch: main
PostHog (MCP) branch: master
Timestamp: 2026-07-07T18:55:27.194Z
Duration: 468.7s

YARA Scanner

✓ 54 tool calls scanned, 0 violations detected

No violations: ✓ 54 clean scans

@wizard-ci-bot

wizard-ci-bot Bot commented Jul 7, 2026

Copy link
Copy Markdown
Author

PR Evaluation Report

Summary

This PR integrates PostHog analytics into a FastAPI AI SaaS application. It adds module-level PostHog initialization in the app lifespan, creates a custom ASGI middleware for per-request user context, and instruments 11 events across 6 router modules covering auth, content generation, API key management, settings, and usage tracking.

Files changed Lines added Lines removed
12 +192 -1

Confidence score: 5/5 🧙

  • Email used as distinct_id: identify_context(user.email) is used throughout instead of str(user.id). The user model has an id field (confirmed by User.get_by_id() and current_user.id usage). Using raw emails as distinct IDs causes fragmented analytics if users change their email and won't match frontend distinct IDs. [CRITICAL]
  • No error tracking: Neither enable_exception_autocapture=True nor any capture_exception() calls are present. The middleware's context will silently swallow exceptions without reporting them to PostHog error tracking. [CRITICAL]
  • posthog.flush() instead of posthog.shutdown(): The lifespan shutdown calls posthog.flush() but should call posthog.shutdown() to properly clean up background threads and ensure all events are sent. [MEDIUM]

File changes

Filename Score Description
app/main.py 4/5 Lifespan-based initialization with PostHog module-level config; missing shutdown()
app/middleware.py 3/5 Clean ASGI middleware with context wrapping, but uses email as distinct_id
app/config.py 4/5 Pydantic Settings with PostHog config; follows existing patterns
app/routers/auth.py 3/5 Auth events well-instrumented but uses email as distinct_id
app/routers/generate.py 5/5 Good event coverage for both success and failure paths
app/routers/api_keys.py 4/5 API key lifecycle events with enriched properties
app/routers/pages.py 4/5 Dashboard view tracking with relevant properties
app/routers/settings.py 4/5 Settings and password change events captured
app/routers/usage.py 4/5 Usage stats viewing tracked with context
.env.example 5/5 PostHog env vars documented
requirements.txt 5/5 PostHog SDK added
posthog-setup-report.md 4/5 Comprehensive setup report

App sanity check ✅

Criteria Result Description
App builds and runs Yes No syntax errors; all imports resolve correctly
Preserves existing env vars & configs Yes Existing config, routes, and middleware untouched
No syntax or type errors Yes All Python syntax is valid; type usage is consistent
Correct imports/exports Yes posthog, capture, identify_context, new_context, tag all imported correctly from the posthog package
Minimal, focused changes Yes All changes are PostHog-related; no scope creep
Pre-existing issues None —

Other completed criteria

  • Environment variables documented in .env.example with POSTHOG_PROJECT_TOKEN and POSTHOG_HOST
  • Build configuration valid — posthog>=3.0.0 added to requirements.txt with valid version specifier
  • Pydantic Settings class extended cleanly with PostHog fields following existing patterns

PostHog implementation ⚠️

Criteria Result Description
PostHog SDKs installed Yes posthog>=3.0.0 in requirements.txt
PostHog client initialized Yes Module-level posthog.api_key / posthog.host set in FastAPI lifespan context manager, following the FastAPI-specific pattern
capture() Yes 11 meaningful capture() calls across all router modules
identify() N/A Server-only app
Error tracking No No enable_exception_autocapture=True, no capture_exception() calls anywhere
Reverse proxy N/A Server-only app

Issues

  • No error tracking configured: The Python SDK rules require enable_exception_autocapture=True to automatically track exceptions. Neither autocapture nor manual capture_exception() calls are present. The middleware wraps requests in new_context() which would capture exceptions if autocapture were enabled, but it isn't. Add posthog.enable_exception_autocapture = True during initialization or switch to the instance-based Posthog() constructor with enable_exception_autocapture=True. [CRITICAL]
  • Email used as distinct_id: identify_context(user.email) is used in the middleware and auth routes. The User model has a proper id field (used as current_user.id throughout the codebase). Using email as distinct_id causes data fragmentation if users change their email and won't link to frontend sessions. Change to identify_context(str(user.id)). [CRITICAL]
  • flush() instead of shutdown(): The lifespan shutdown calls posthog.flush() but should call posthog.shutdown() to flush the queue and cleanly terminate background threads. [MEDIUM]

Other completed criteria

  • API key loaded from environment variable via Pydantic Settings (posthog_project_token)
  • Host correctly configured, defaulting to https://us.i.posthog.com
  • posthog_disabled flag allows disabling PostHog in test/dev environments
  • Middleware correctly wraps requests in new_context() for context propagation
  • posthog.set() correctly used for person properties (not identify() which doesn't exist in the Python SDK)

PostHog insights and events ⚠️

Filename PostHog events Description
app/routers/auth.py user_signed_up, user_logged_in, user_logged_out Full auth lifecycle; signup includes initial_credits, login includes login_method
app/routers/generate.py content_generated, generation_failed_insufficient_credits Core product action with success/failure paths; includes credits_used, generation_type, prompt_length
app/routers/api_keys.py api_key_created, api_key_revoked API key lifecycle tracking; creation includes active_key_count
app/routers/pages.py dashboard_viewed Page view with total_generations, credits_remaining, api_key_count
app/routers/settings.py settings_updated, password_changed Account management events; settings_updated includes fields_changed
app/routers/usage.py usage_stats_viewed Usage page tracking with days_range and total_generations

Issues

  • api_key_revoked lacks properties: The api_key_revoked event has no properties — it would benefit from at least the key name or remaining active key count for context. [LOW]

Other completed criteria

  • Events represent real user actions mapping to actual product flows (signup → generate content → view usage)
  • Events enable product insights — can build signup-to-generation funnel, track insufficient credits as churn signal, monitor API key usage
  • Events include relevant enriched properties (credits_used, generation_type, prompt_length, etc.)
  • No PII in event properties — emails are not passed in capture() properties; person properties set via posthog.set() correctly
  • Event names are descriptive, consistent snake_case, and follow [noun]_[verb] convention

Reviewed by wizard workbench PR evaluator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants